Non-image-forming vision as measured through ipRGC-mediated pupil constriction is not modulated by covert visual attention.

In brightness, the pupil constricts, while in darkness, the pupil dilates; this is known as the pupillary light response (PLR). The PLR is driven by all photoreceptors: rods and cones, which contribute to image-forming vision, and intrinsically photosensitive retinal ganglion cells (ipRGCs), which mainly contribute to non-image-forming vision. Rods and cones cause immediate pupil constriction upon light exposure, whereas ipRGCs cause sustained constriction throughout light exposure. Recent studies have shown that covert attention modulated the initial PLR; however, it remains unclear whether the same holds for the sustained PLR. We tested this by leveraging ipRGCs' responsiveness to blue light, causing the most prominent sustained constriction. While replicating previous studies by showing that pupils constricted more when either directly looking at, or covertly attending to, bright as compared to dim stimuli (with the same color), we also found that the pupil constricted more when directly looking at blue as compared to red stimuli (with the same luminosity). Crucially, however, in two high-powered studies (n = 60), we did not find any pupil-size difference when covertly attending to blue as compared to red stimuli. This suggests that ipRGC-mediated pupil constriction, and possibly non-image-forming vision more generally, is not modulated by covert attention.

Functional benefits of cognitively driven pupil-size changes.

Pupil-size changes are typically associated with the pupil light response (PLR), where they are driven by the physical entry of light into the eye. However, pupil-size changes are also influenced by various cognitive processes, where they are driven by higher-level cognition. For example, the strength of the PLR is not solely affected by physical properties of the light but also by cognitive factors, such as whether the source of light is attended or not, which results in an increase or decrease in the strength of the PLR. Surprisingly, although cognitively driven pupil-size changes have been the focus of extensive research, their possible functions are rarely discussed. Here we consider the relative (dis)advantages of small versus large pupils in different situations from a theoretical point of view, and compare these to empirical results showing how pupil size actually changes in these situations. Based on this, we suggest that cognitively driven pupil-size changes optimize vision either through preparation, embodied representations, or a differential emphasis on central or peripheral vision. More generally, we argue that cognitively driven pupil-size changes are a form of sensory tuning: a subtle adjustment of the eyes to optimize vision for the current situation and the immediate future. This article is categorized under: Neuroscience > Cognition Neuroscience > Physiology Neuroscience > Behavior.

Revealing visual working memory operations with pupillometry: Encoding, maintenance, and prioritization.

Pupillary dynamics reflect effects of distinct and important operations of visual working memory: encoding, maintenance, and prioritization. Here, we review how pupil size predicts memory performance and how it provides novel insights into the mechanisms of each operation. Visual information must first be encoded into working memory with sufficient precision. The depth of this encoding process couples to arousal-linked baseline pupil size as well as a pupil constriction response before and after stimulus onset, respectively. Subsequently, the encoded information is maintained over time to ensure it is not lost. Pupil dilation reflects the effortful maintenance of information, wherein storing more items is accompanied by larger dilations. Lastly, the most task-relevant information is prioritized to guide upcoming behavior, which is reflected in yet another dilatory component. Moreover, activated content in memory can be pupillometrically probed directly by tagging visual information with distinct luminance levels. Through this luminance-tagging mechanism, pupil light responses reveal whether dark or bright items receive more attention during encoding and prioritization. Together, conceptualizing pupil responses as a sum of distinct components over time reveals insights into operations of visual working memory. From this viewpoint, pupillometry is a promising avenue to study the most vital operations through which visual working memory works. This article is categorized under: Psychology > Attention Psychology > Memory Psychology > Theory and Methods.

Attention-based rehearsal: Eye movements reveal how visuospatial information is maintained in working memory.

The human eye scans visual information through scan paths, series of fixations. Analogous to these scan paths during the process of actual "seeing," we investigated whether similar scan paths are also observed while subjects are "rehearsing" stimuli in visuospatial working memory. Participants performed a continuous recall task in which they rehearsed the precise location and color of three serially presented discs during a retention interval, and later reproduced either the precise location or the color of a single probed item. In two experiments, we varied the direction along which the items were presented and investigated whether scan paths during rehearsal followed the pattern of stimulus presentation during encoding (left-to-right in Experiment 1; left-to-right/right-to-left in Experiment 2). In both experiments, we confirmed that the eyes follow similar scan paths during encoding and rehearsal. Specifically, we observed that during rehearsal participants refixated the memorized locations they saw during encoding. Most interestingly, the precision with which these locations were refixated was associated with smaller recall errors. Assuming that eye position reflects the focus of attention, our findings suggest a functional contribution of spatial attention shifts to working memory and are in line with the hypothesis that maintenance of information in visuospatial working memory is supported by attention-based rehearsal. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Effects of pupil size as manipulated through ipRGC activation on visual processing.

The size of the eyes' pupils determines how much light enters the eye and also how well this light is focused. Through this route, pupil size shapes the earliest stages of visual processing. Yet causal effects of pupil size on vision are poorly understood and rarely studied. Here we introduce a new way to manipulate pupil size, which relies on activation of intrinsically photosensitive retinal ganglion cells (ipRGCs) to induce sustained pupil constriction. We report the effects of both experimentally induced and spontaneous changes in pupil size on visual processing as measured through EEG. We compare these to the effects of stimulus intensity and covert visual attention, because previous studies have shown that these factors all have comparable effects on some common measures of early visual processing, such as detection performance and steady-state visual evoked potentials; yet it is still unclear whether these are superficial similarities, or rather whether they reflect similar underlying processes. Using a mix of neural-network decoding, ERP analyses, and time-frequency analyses, we find that induced pupil size, spontaneous pupil size, stimulus intensity, and covert visual attention all affect EEG responses, mainly over occipital and parietal electrodes, but-crucially-that they do so in qualitatively different ways. Induced and spontaneous pupil-size changes mainly modulate activity patterns (but not overall power or intertrial coherence) in the high-frequency beta range; this may reflect an effect of pupil size on oculomotor activity and/ or visual processing. In addition, spontaneous (but not induced) pupil size tends to correlate positively with intertrial coherence in the alpha band; this may reflect a non-causal relationship, mediated by arousal. Taken together, our findings suggest that pupil size has qualitatively different effects on visual processing from stimulus intensity and covert visual attention. This shows that pupil size as manipulated through ipRGC activation strongly affects visual processing, and provides concrete starting points for further study of this important yet understudied earliest stage of visual processing.

Emphasis on peripheral vision is accompanied by pupil dilation.

People are best able to detect stimuli in peripheral vision when their pupils are large, and best able to discriminate stimuli in central vision when their pupils are small. However, it is unclear whether our visual system makes use of this by dilating the pupils when attention is directed towards peripheral vision. Therefore, throughout three experiments (N = 100), we tested whether pupil size adapts to the "breadth" of attention. We found that pupils dilate with increasing attentional breadth, both when attention is diffusely spread and when attention is directed at specific locations in peripheral vision. Based on our results and others, we propose that cognitively driven pupil dilation is not an epiphenomenal marker of locus coeruleus activity, as is often assumed, but rather is an adaptive response that reflects an emphasis on peripheral vision.

Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis.

Cognitive pupillometry is the measurement of pupil size to investigate cognitive processes such as attention, mental effort, working memory, and many others. Currently, there is no commonly agreed-upon methodology for conducting cognitive-pupillometry experiments, and approaches vary widely between research groups and even between different experiments from the same group. This lack of consensus makes it difficult to know which factors to consider when conducting a cognitive-pupillometry experiment. Here we provide a comprehensive, hands-on guide to methods in cognitive pupillometry, with a focus on trial-based experiments in which the measure of interest is the task-evoked pupil response to a stimulus. We cover all methodological aspects of cognitive pupillometry: experimental design, preprocessing of pupil-size data, and statistical techniques to deal with multiple comparisons when testing pupil-size data. In addition, we provide code and toolboxes (in Python) for preprocessing and statistical analysis, and we illustrate all aspects of the proposed workflow through an example experiment and example scripts.

Is Categorization in Visual Working Memory a Way to Reduce Mental Effort? A Pupillometry Study.

Recent studies on visual working memory (VWM) have shown that visual information can be stored in VWM as continuous (e.g., a specific shade of red) as well as categorical representations (e.g., the general category red). It has been widely assumed, yet never directly tested, that continuous representations require more VWM mental effort than categorical representations; given limited VWM capacity, this would mean that fewer continuous, as compared to categorical, representations can be maintained simultaneously. We tested this assumption by measuring pupil size, as a proxy for mental effort, in a delayed estimation task. Participants memorized one to four ambiguous (boundaries between adjacent color categories) or prototypical colors to encourage continuous or categorical representations, respectively; after a delay, a probe indicated the location of the to-be-reported color. We found that, for memory load 1, pupil size was larger while maintaining ambiguous as compared to prototypical colors, but without any difference in memory precision; this suggests that participants relied on an effortful continuous representation to maintain a single ambiguous color, thus resulting in pupil dilation while preserving precision. Strikingly, this effect gradually inverted, such that for memory load 4, pupil size was smaller while maintaining ambiguous and prototypical colors, but memory precision was now substantially reduced for ambiguous colors; this suggests that with increased memory load participants increasingly relied on categorical representations for ambiguous colors (which are by definition a poor fit to any category). Taken together, our results suggest that continuous representations are more effortful than categorical representations and that very few continuous representations (perhaps only one) can be maintained simultaneously.

Interactions Between Visual Working Memory, Attention, and Color Categories: A Pupillometry Study.

Recent studies have found that visual working memory (VWM) for color shows a categorical bias: observers typically remember colors as more prototypical to the category they belong to than they actually are. Here, we further examine color-category effects on VWM using pupillometry. Participants remembered a color for later reproduction on a color wheel. During the retention interval, a colored probe was presented, and we measured the pupil constriction in response to this probe, assuming that the strength of constriction reflects the visual saliency of the probe. We found that the pupil initially constricted most strongly for non-matching colors that were maximally different from the memorized color; this likely reflects a lack of visual adaptation for these colors, which renders them more salient than memory-matching colors (which were shown before). Strikingly, this effect reversed later in time, such that pupil constriction was more prolonged for memory-matching colors as compared to non-matching colors; this likely reflects that memory-matching colors capture attention more strongly, and perhaps for a longer time, than non-matching colors do. We found no effects of color categories on pupil constriction: after controlling for color distance, (non-matching) colors from the same category as the memory color did not result in a different pupil response as compared to colors from a different category; however, we did find that behavioral responses were biased by color categories. In summary, we found that pupil constriction to colored probes reflects both visual adaptation and VWM content, but, unlike behavioral measures, is not notably affected by color categories.

Categorical bias as a crucial parameter in visual working memory: The effect of memory load and retention interval.

Visual information can be stored as continuous as well as categorical representations in visual working memory (VWM) to guide subsequent behavior. Yet it is still unclear what determines whether VWM is represented as continuous or categorical information, or as a mix of both. Recent studies have shown that color VWM representations adjust flexibly depending on the number of memory items as well as the duration that these items need to be maintained for. The current study aims to extend and replicate these crucial effects. In a delayed estimation task, participants memorized one to four colored objects presented at different spatial locations, followed by a delay of 100, 500, 1000, or 2000 msec. Next, a probe indicated the location of the color that participants needed to report. We measured the extent to which responses were biased in the direction of prototypical colors. Crucially, we implemented this categorical bias in an extension to the classic mixture model (Zhang & Luck, 2008) in which the center of the error distribution is a crucial parameter that characterizes the extent to which VWM is biased by color categories. We found that VWM shows a strong categorical bias in all cases, and that this bias increases with increasing memory load; strikingly, this effect of memory load on categorical bias is stronger at longer intervals (1000 msec or longer), as compared to shorter intervals, yet it peaks for intermediate memory loads as opposed to the highest memory load. Overall, our results suggest that when visual information needs to be maintained for one second or longer, VWM becomes more reliant on categorical representations as memory load increases.

Coordination effort in joint action is reflected in pupil size.

Humans often perform visual tasks together, and when doing so, they tend to devise division of labor strategies to share the load. Implementing such strategies, however, is effortful as co-actors need to coordinate their actions. We tested if pupil size - a physiological correlate of mental effort - can detect such a coordination effort in a multiple object tracking task (MOT). Participants performed the MOT task jointly with a computer partner and either devised a division of labor strategy (main experiment) or the labor division was already pre-determined (control experiment). We observed that pupil sizes increase relative to performing the MOT task alone in the main experiment while this is not the case in the control experiment. These findings suggest that pupil size can detect a rise in coordination effort, extending the view that pupil size indexes mental effort across a wide range of cognitively demanding tasks.

Concurrent guidance of attention by multiple working memory items: Behavioral and computational evidence.

During visual search, task-relevant representations in visual working memory (VWM), known as attentional templates, are assumed to guide attention. A current debate concerns whether only one (Single-Item-Template hypothesis; SIT) or multiple (Multiple-Item-Template hypothesis; MIT) items can serve as attentional templates simultaneously. The current study was designed to test these two hypotheses. Participants memorized two colors, prior to a visual-search task in which the target and the distractor could match or not match the colors held in VWM. Robust attentional guidance was observed when one of the memory colors was presented as the target (reduced response times (RTs) on target-match trials) or the distractor (increased RTs on distractor-match trials). We constructed two drift-diffusion models that implemented the MIT and SIT hypotheses, which are similar in their predictions about overall RTs, but differ in their predictions about RTs on individual trials. Critically, simulated RT distributions and error rates revealed a better match of the MIT hypothesis to the observed data than the SIT hypothesis. Taken together, our findings provide behavioral and computational evidence for the concurrent guidance of attention by multiple items in VWM.

Tuning the Senses: How the Pupil Shapes Vision at the Earliest Stage.

The pupil responds reflexively to changes in brightness and focal distance to maintain the smallest pupil (and thus the highest visual acuity) that still allows sufficient light to reach the retina. The pupil also responds to a wide variety of cognitive processes, but the functions of these cognitive responses are still poorly understood. In this review, I propose that cognitive pupil responses, like their reflexive counterparts, serve to optimize vision. Specifically, an emphasis on central vision over peripheral vision results in pupil constriction, and this likely reflects the fact that central vision benefits most from the increased visual acuity provided by small pupils. Furthermore, an intention to act with a bright stimulus results in preparatory pupil constriction, which allows the pupil to respond quickly when that bright stimulus is subsequently brought into view. More generally, cognitively driven pupil responses are likely a form of sensory tuning: a subtle adjustment of the eyes to optimize their properties for the current situation and the immediate future.

Free-viewing multi-stimulus eye tracking task to index attention bias for alcohol versus soda cues: Satisfactory reliability and criterion validity.

Cognitive -motivational models point to attention bias (AB) as an important factor in the persistence of problematic drinking behavior. Unfortunately, the measures that have been used to examine AB in addiction typically showed poor psychometric properties. To bring research on AB a critical step further it would be crucial to develop tasks with acceptable reliability and construct validity. Recently, Lazarov and colleagues (2016) developed a multi-stimulus free-viewing task (participants were free to look at any part of the screen and there was no secondary task involved) that showed excellent psychometric properties in the context of social anxiety as well as depression. We, therefore, adapted this task and examined its psychometric quality within the context of alcohol use. Participants with varying levels of alcohol use (N = 100) were presented with 54 matrices each containing 8 alcoholic and 8 non-alcoholic drinks. Each matrix was presented for 6 s. First fixation (100 ms) location and latency and total dwell time were assessed for alcohol and soda pictures. Assessment of AB, craving, and alcohol use (problems) was repeated after 3-8 days. Specifically, the dwell-time based AB-measure showed excellent internal reliability and considerable stability. Supporting the validity of the current AB-measures, it was found that participants with higher scores on craving and alcohol problems (i) dwelt longer on alcohol stimuli, and (ii) more often showed a first fixation on alcohol, whereas (iii) stronger craving was associated with shorter latency of first alcohol fixations. The AB-measure showed promising psychometric properties. Thus, this free-viewing eye-tracking task seems a welcome new tool for being used in future research on AB in addiction.

Correction to: Can you have multiple attentional templates? Large-scale replications of Van Moorselaar, Theeuwes, and Olivers (2014) and Hollingworth and Beck (2016).

The article was updated to correct a mismatch in the images between the PDF and online versions of the article.

The effect of pupil size and peripheral brightness on detection and discrimination performance.

It is easier to read dark text on a bright background (positive polarity) than to read bright text on a dark background (negative polarity). This positive-polarity advantage is often linked to pupil size: A bright background induces small pupils, which in turn increases visual acuity. Here we report that pupil size, when manipulated through peripheral brightness, has qualitatively different effects on discrimination of fine stimuli in central vision and detection of faint stimuli in peripheral vision. Small pupils are associated with improved discrimination performance, consistent with the positive-polarity advantage, but only for very small stimuli that are at the threshold of visual acuity. In contrast, large pupils are associated with improved detection performance. These results are likely due to two pupil-size related factors: Small pupils increase visual acuity, which improves discrimination of fine stimuli; and large pupils increase light influx, which improves detection of faint stimuli. Light scatter is likely also a contributing factor: When a display is bright, light scatter creates a diffuse veil of retinal illumination that reduces perceived image contrast, thus impairing detection performance. We further found that pupil size was larger during the detection task than during the discrimination task, even though both tasks were equally difficult and similar in visual input; this suggests that the pupil may automatically assume an optimal size for the current task. Our results may explain why pupils dilate in response to arousal: This may reflect an increased emphasis on detection of unpredictable danger, which is crucially important in many situations that are characterized by high levels of arousal. Finally, we discuss the implications of our results for the ergonomics of display design.

The pupillary light response reflects visual working memory content.

Recent studies have shown that the pupillary light response (PLR) is modulated by higher cognitive functions, presumably through activity in visual sensory brain areas. Here we use the PLR to test the involvement of sensory areas in visual working memory (VWM). In two experiments, participants memorized either bright or dark stimuli. We found that pupils were smaller when a prestimulus cue indicated that a bright stimulus should be memorized; this reflects a covert shift of attention during encoding of items into VWM. Crucially, we obtained the same result with a poststimulus cue, which shows that internal shifts of attention within VWM affect pupil size as well. Strikingly, the effect of VWM content on pupil size was most pronounced immediately after the poststimulus cue, and then dissipated. This suggests that a shift of attention within VWM momentarily activates an "active" memory representation, but that this representation quickly transforms into a "hidden" state that does not rely on sensory areas. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Can you have multiple attentional templates? Large-scale replications of Van Moorselaar, Theeuwes, and Olivers (2014) and Hollingworth and Beck (2016).

Stimuli that resemble the content of visual working memory (VWM) capture attention. However, theories disagree on how many VWM items can bias attention simultaneously. According to some theories, there is a distinction between active and passive states in VWM, such that only items held in an active state can bias attention. The single-item-template hypothesis holds that only one item can be in an active state and thus can bias attention. In contrast, the multiple-item-template hypothesis posits that multiple VWM items can be in an activate state simultaneously, and thus can bias attention. Recently, Van Moorselaar, Theeuwes, and Olivers (Journal of Experimental Psychology: Human Perception and Performance, 40(4):1450, 2014) and Hollingworth and Beck (Journal of Experimental Psychology: Human Perception and Performance, 42(7):911-917, 2016) tested these accounts, but obtained seemingly contradictory results. Van Moorselaar et al. (2014) found that a distractor in a visual-search task captured attention more when it matched the content of VWM (memory-driven capture). Crucially, memory-driven capture disappeared when more than one item was held in VWM, in line with the single-item-template hypothesis. In contrast, Hollingworth and Beck (2016) found memory-driven capture even when multiple items were kept in VWM, in line with the multiple-item-template hypothesis. Considering these mixed results, we replicated both studies with a larger sample, and found that all key results are reliable. It is unclear to what extent these divergent results are due to paradigm differences between the studies. We conclude that is crucial to our understanding of VWM to determine the boundary conditions under which memory-driven capture occurs.